Camera activation response to vehicle safety event

ABSTRACT

Methods and systems are provided for transmitting video data associated with a vehicle. In one exemplary embodiment, a vehicle processor determines a vehicle safety event based on one or more safety sensors, activates one or more vehicle cameras that capture video data in response to the determined vehicle safety event, records the video data in a vehicle data storage device, and, if the vehicle processor determines that a suitable connection to a telecommunications network is available when the video data is being captured, then transmits the video data over the telecommunications network in substantially real time.

TECHNICAL FIELD

The present disclosure generally relates to vehicles, and moreparticularly relates to methods and systems for transmitting video datausing one or more cameras of the vehicle.

BACKGROUND

Theft, acts of vandalism, and other events can occur to a vehicle. Also,a vehicle owner may, in certain circumstances, desire an enhanced feelof security for the vehicle.

Accordingly, it may be desirable for a vehicle owner, the relevantauthorities and other parties to be able to ascertain the cause of avehicle event, or the culprits of theft and vandalism. In the case ofthefts and vandalism, it would be desirable to prevent such actswherever possible or at least provide a tool to assist such. Further, itmay be desirable to provide a tool to allow an owner to be reassured onthe security of a vehicle.

The present disclosure provides methods, systems and vehicles forachieving one or more of these desires in various embodiments and otherrelated desires in various embodiments. Furthermore, other desirablefeatures and characteristics of the present invention will becomeapparent from the subsequent detailed description of the invention andthe appended claims, taken in conjunction with the accompanying drawingsand this background of the invention.

SUMMARY

In accordance with an exemplary embodiment, a method of transmittingvideo data associated with a vehicle is provided. The method comprisesthe steps of determining, via a vehicle processor, a vehicle safetyevent based on one or more safety sensors, activating, via the vehicleprocessor, one or more vehicle cameras that capture video data inresponse to the determined vehicle safety event, recording, via thevehicle processor, the video data in a vehicle data storage device, and,if the vehicle processor determines that a suitable connection to atelecommunications network is available when the video data is beingcaptured, then transmitting, via the vehicle processor, the video dataover the telecommunications network in substantially real time.

In accordance with another exemplary embodiment, a vehicle controlsystem for transmitting video data associated with a vehicle isprovided. The control system comprises one or more safety sensors and avehicle processor. The one or more safety sensors are responsive to avehicle safety event. The vehicle processor is configured to activateone or more vehicle cameras to capture video data in response to thevehicle safety event, record the video data in a vehicle data storagedata device, and, if the processor determines that a suitabletelecommunications network connection is available when the video datais being captured, then the vehicle processor is configured to transmitthe video data over the telecommunications network in substantially realtime.

In accordance with a further exemplary embodiments, a vehicle isprovided. The vehicle comprises one or more vehicle cameras, one or moresafety sensors responsive to a vehicle safety event, a vehicle datastorage data device, and a vehicle control system. The vehicle controlsystem is for transmitting video data associated with the vehicle, andcomprises a vehicle processor configured to activate the one or morevehicle cameras to capture video data in response to the determinedvehicle safety event, record the video data in the vehicle data storagedata device, and, if the processor determines that a suitabletelecommunications network connection is available when the video datais being captured, then the vehicle processor is configured to transmitthe video data over the telecommunications network in substantially realtime.

DESCRIPTION OF THE DRAWINGS

The present disclosure will hereinafter be described in conjunction withthe following drawing figures, wherein like numerals denote likeelements, and wherein:

FIG. 1 is a functional block diagram of a vehicle that includes one ormore vehicle cameras and a remote communications module forcommunicating video data to a remote server or remote user device, inaccordance with an exemplary embodiment;

FIG. 2 is a functional block diagram of a network of a vehicle, a remoteserver and a remote user device, in accordance with an exemplaryembodiment;

FIG. 3 is a functional block diagram of a vehicle control system, aremote server control system and a remote user device control system, inaccordance with an exemplary embodiment;

FIG. 4 is a flowchart of a process for transmitting video data to aremote server or a remote user device and activating or deactivating thetransmission of video data by one or more vehicle control commands fromthe remote server or the remote user device, in connection with thevehicle of FIG. 1, in accordance with an exemplary embodiment;

FIG. 5 is a flowchart of a process for transmitting video data to aremote server or a remote user device and transmitting one or morevehicle control commands from the remote server or the remote userdevice to change a parameter of the video data or the one or morevehicle cameras in connection with the vehicle of FIG. 1, in accordancewith an exemplary embodiment;

FIG. 6 is a functional block diagram of a graphical user (operator)interface for selecting control commands;

FIG. 7 is a flowchart of a process for transmitting video data inresponse to a trigger signal, in connection with the vehicle of FIG. 1,in accordance with an exemplary embodiment.

FIG. 8 is a schematic diagram of an in-vehicle video data transmissionbutton arrangement, in accordance with an exemplary embodiment.

FIG. 9 is a schematic diagram of a key fob button arrangement or agraphic user interface button arrangement including a vehicle datatransmission button, in accordance with an exemplary embodiment; and

FIG. 10 is a flowchart of a process for transmitting video data to aremote server for viewing by an operator, in connection with the vehicleof FIG. 1, in accordance with an exemplary embodiment;

DETAILED DESCRIPTION

The following detailed description is merely exemplary in nature and isnot intended to limit the disclosure or the application and usesthereof. Furthermore, there is no intention to be bound by any theorypresented in the preceding background or the following detaileddescription.

FIG. 1 illustrates a vehicle 100, or automobile, according to anexemplary embodiment. The vehicle 100 may be any one of a number ofdifferent types of automobiles, such as, for example, a sedan, a wagon,a truck, or a sport utility vehicle (SUV), and may be two-wheel drive(2WD) (i.e., rear-wheel drive or front-wheel drive), four-wheel drive(4WD) or all-wheel drive (AWD).

As described in greater detail further below, the vehicle 100 includesvarious doors 101 as well as a control system 102 for controlling videodata, particularly the transmission thereof. In the depicted embodiment,the doors 101 include side doors 110 on the driver's side andpassenger's side of the vehicle 100 as well as a rear door 111 in a rearportion (or rear region) 146 of the vehicle 100. In one embodiment, therear door 111 comprises a rear hatch for the vehicle 100. In otherembodiments, the rear door 111 may comprise a trunk door and/or othertype of rear door. It will be appreciated that the number and/orconfiguration of doors 101 may vary in different embodiments.

Also as discussed further below, the control system 102 includes one ormore cameras 103, a sensor array 104, a controller 106, and a displaysystem 108 (also referred to herein as a display unit). In variousembodiments, the control system 102 is configured to control the one ormore cameras 103 and to control transmission of video data captured bythe one or more cameras 103.

In one embodiment depicted in FIG. 1, vehicle 100 includes, in additionto the above-referenced doors 101, rear region 146, and control system102, a chassis 112, a body 114, four wheels 116, an electronic system118, a powertrain 129, a rear view mirror 140, side mirrors 142, a frontgrill 144, an infotainment system 148 (e.g. radio, video, navigation,and/or other system providing information and/or entertainment for auser of the vehicle 100) a steering system 150, a braking system 155,and one or more other driver input systems 160. The body 114 is arrangedon the chassis 112 and substantially encloses the other components ofthe vehicle 100. The body 114 and the chassis 112 may jointly form aframe. The wheels 116 are each rotationally coupled to the chassis 112near a respective corner of the body 114. As depicted in FIG. 1, eachwheel 116 comprises a wheel assembly that includes a tire as well as awheel and related components (and that are collectively referred to asthe “wheel 116” for the purposes of this Application). In variousembodiments the vehicle 100 may differ from that depicted in FIG. 1. Forexample, in certain embodiments the number of wheels 116 may vary. Byway of additional example, in various embodiments the vehicle 100 maynot have a steering system, and for example may be steered bydifferential braking, among various other possible differences.

In the exemplary embodiment illustrated in FIG. 1, the powertrain 129includes an actuator assembly 120 that includes an engine 130. Invarious other embodiments, the powertrain 129 may vary from thatdepicted in FIG. 1 and/or described below (e.g. in some embodiments thepowertrain may include a gas combustion engine 130, while in otherembodiments the powertrain 129 may include an electric motor, alone orin combination with one or more other powertrain 129 components, forexample for electric vehicles, hybrid vehicles, and the like). In oneembodiment depicted in FIG. 1, the actuator assembly 120 and thepowertrain 129 are mounted on the chassis 112 that drives the wheels116. In one embodiment, the engine 130 comprises a combustion engine,and is housed in an engine mounting apparatus 131. In various otherembodiments, the engine 130 may comprise an electric motor and/or one ormore other transmission system 129 components (e.g. for an electricvehicle).

It will be appreciated that in other embodiments, the actuator assembly120 may include one or more other types of engines and/or motors, suchas an electric motor/generator, instead of or in addition to thecombustion engine. In certain embodiments, the electronic system 118comprises an engine system that controls the engine 130 and/or one ormore other systems of the vehicle 100.

Still referring to FIG. 1, in one embodiment, the engine 130 is coupledto at least some of the wheels 116 through one or more drive shafts 134.In some embodiments, the engine 130 is mechanically coupled to thetransmission. In other embodiments, the engine 130 may instead becoupled to a generator used to power an electric motor that ismechanically coupled to the transmission. In certain other embodiments(e.g. electrical vehicles), an engine and/or transmission may not benecessary.

The steering system 150 is mounted on the chassis 112, and controlssteering of the wheels 116. In one embodiment, the steering system mayinclude a non-depicted steering wheel and a steering column. In variousembodiments, the steering wheel receives inputs from a driver of thevehicle 100, and the steering column results in desired steering anglesfor the wheels 116 via the drive shafts 134 based on the inputs from thedriver. In certain embodiments, an autonomous vehicle may utilizesteering commands that are generated by a computer, with no involvementfrom the driver.

The braking system 155 is mounted on the chassis 112, and providesbraking for the vehicle 100. The braking system 155 receives inputs fromthe driver via a non-depicted brake pedal, and provides appropriatebraking via brake units (not depicted).

Other driver input systems 160 may include an acceleration input systemcomprising an accelerator pedal 161 that is engaged by a driver, withthe engagement representative of a desired speed or acceleration of thevehicle 100. The other driver input systems 160 may also include, amongother possible systems, various other inputs for various vehicle devicesand/or systems, such as for the infotainment system 148, and/or one ormore environmental systems, lighting units, and the like (not depicted).Similar to the discussion above regarding possible variations for thevehicle 100, in certain embodiments steering, braking, suspension,acceleration, and/or other driving features can be commanded by acomputer instead of by a driver.

In one embodiment, the control system 102 is mounted on the chassis 112.As discussed above, the control system 102 provided is configured tocontrol various aspects of operation of the one or more cameras 103 andtransmission, e.g. streaming, of video data captured by the one or morecameras 103.

As noted above and depicted in FIG. 1, in one embodiment the controlsystem 102 comprises a plurality of cameras 103, a sensor array 104, acontroller 106, a remote communications module 202, and a display system108. While the components of the control system 102 (including thecameras 103, the sensor array 104, the controller 106, and the displaysystem 108) are depicted as being part of the same system, it will beappreciated that in certain embodiments these features may comprise twoor more systems. In addition, in various embodiments the control system102 may comprise all or part of, and/or may be coupled to, various othervehicle devices and systems, such as, among others, the actuatorassembly 120, the electronic system 118, and/or one or more othersystems of the vehicle 100.

The plurality of cameras 103 are operable to obtain video data withrespect to various different locations associated with the vehicle 100.As depicted in one embodiment, cameras 103 are included within orproximate each of the rear view mirror 140, side mirrors 142, frontgrill 144, and rear region 146 (e.g. trunk 147 or rear door/hatch 111).In one embodiment, the cameras 103 comprise video cameras controlled viathe controller 106. In various embodiments, the cameras 103 may also bedisposed in or proximate one or more other locations of the vehicle 100.For example, more or less than four cameras 103 could be included, suchas six cameras 103 for capturing video data of the vehicle's outsidesurroundings. Although the one or more cameras 103 are shown forcapturing video data of the outside of the vehicle 100, the one or morecameras 103 can include one or more cameras 103 for capturing video dataof the interior of the vehicle 100. Such one or more interior cameras103 may be arranged for capturing videos data of the driver of thevehicle 100 and optionally also at least one of a front passenger andany rear passengers.

In various embodiments, the vehicle 100 comprises one or moremicrophones 289 for picking up interior and/or exterior sound. The oneor more microphones 289 may be included as part of the one or morecameras 103 or may be separate devices.

In some embodiments, the one or more cameras 103 are each associatedwith a camera washer or wiper 280. The camera washer or wiper 280 maycomprise a blade for wiping a lens or other front optic of the camera103. Alternatively, the camera washer or wiper 280 may comprise a nozzleand be associated with a pump for directing cleaning fluid (e.g. water)onto a front optic of the camera 103.

In some embodiments, the one or more cameras 103 are each associatedwith an articulator 282 for articulating the camera 103 to change thefield of view. The articulator may comprise a motor (not shown), atransmission and a pivot to allow the camera 103 to change angle.

In various embodiments, the vehicle 100 is equipped with one or moreillumination devices 284, 286. The one or more illumination devices 284,286 may comprise one or more exterior illumination devices 284 forilluminating outside of the vehicle 100 or one or more interiorillumination devices 286 for illuminating inside of the vehicle 100. Theone or more illumination devices 284, 286 may comprise the frontheadlamps of the vehicle and/or rear taillights and/or indicator lightsand/or any other lights already required in vehicle design.Alternatively, or additionally, the one or more illumination devices284, 286 may be dedicated to the function of illuminating a field ofview of the one or more cameras 103. Exemplary dedicated illuminationdevices 284, 286 may be located adjacent or co-located with the one ormore cameras 103. The vehicle 100 may comprise front, rear and/or sideillumination devices 284, 286 corresponding to front, rear and/or sidecameras 103.

In various embodiments, the vehicle 100 comprises one or more sounddevices 288, e.g. speakers 288. The one or more speakers 288 inconjunction with the vehicle control system 102, are configured to soundan alarm or horn. The alarm may be a siren like sound or a continuoussound. Alternatively, speech may be sounded to provide a comprehendiblewarning, such as a repeat warning of “stay away from the vehicle, youare being video monitored, police have been notified”.

The sensor array 104 includes various sensors (also referred to hereinas sensor units) that are used for providing measurements and/or datafor use by the controller 106. In various embodiments, the sensors ofthe sensor array 104 comprise one or more detection sensors 162, eventsensors 167, one or more security sensors 166, interface sensors 163,gear sensors 164, and/or wheel speed sensors 165. The detection sensors162 (e.g. radar, lidar, sonar, machine vision, Hall Effect, and/or othersensors) detect objects in proximity to the vehicle 100. The interfacesensors 163 detect a user's engagement of an interface of the vehicle100 (e.g. a button, a knob, a display screen, and/or one or more otherinterfaces), for example in initiating a request for a display of thedisplay system 108 with respect to the doors 101 of the vehicle 100. Thegear sensors 164 detect a gear or transmission state of the vehicle 100(e.g. park, drive, neutral, or reverse). The wheel speed sensors 165measure a speed of one or more of the wheels 116 of the vehicle 100. Itwill be appreciated that in certain embodiments the cameras 103 may beconsidered as part of the sensor array 104. In various embodiments, theone or more security sensors 166 provide a sense signal relating topossible theft or vandalism of the vehicle 100. For example, the one ormore security sensors 166 may include a force sensor such as a vibrationsensor, an accelerometer or some other motion sensor for sensing thatthe vehicle 100 is the subject of a force possibly indicative of a theftevent. The one or more security sensors 166 may comprise a sensor forindicating a key copy, which is an unauthorized copy of a key forunlocking the vehicle doors 101. The one or more security sensors 166may comprise a door sensor for indicating that one or more of thevehicle doors 101 are open. Other sensors for indicating the security ofthe vehicle 100 with respect to theft and vandalism could also beincluded in the one or more security sensors 166.

In various embodiments, the one or more event sensors 167 comprise anyone or more of at least one acceleration sensor, at least one impactsensor, at least one force sensor, at least one roll over sensor, and atleast one pressure sensor.

The controller 106 is coupled to the remote communications module 202.The remote communications module 202 is configured to transmit, e.g.stream video captured by the one or more cameras 103 to a remote server900 or to a remote user device 800 via a cellular network 1000, as willbe discussed in further detail below. In some embodiments, thecontroller 106 is configured to transmit, e.g. stream, sound datacaptured by the one or more microphones 289. The sound data may betransmitted along with the video data to allow synchronized sound andvideo replay at the remote server 900 or the remote user device 800. Theremote communications module 202 may comprise a control unit (not shown)and a transceiver (not shown) for transmitting and receiving data overthe cellular network 1000. The remote communications module 202 mayoperate through third generation (3G) or fourth generation (4G)standards or any other telecommunications standard.

The controller 106 is coupled to the cameras 103, the sensor array 104,the remote communications module 202 and the display system 108. Thecontroller 106 is configured to utilize the various measurements andinformation from the sensor array 104, particularly the one or moresecurity sensors 166, and is configured to operate the one or morecameras 103 and stream captured video data via the remote communicationsmodule 202 in response thereto.

As depicted in FIG. 1, the controller 106 comprises a computer system.In certain embodiments, the controller 106 may also include one or moreof the sensors of the sensor array 104, one or more other devices and/orsystems, and/or components thereof. In addition, it will be appreciatedthat the controller 106 may otherwise differ from the embodimentdepicted in FIG. 1. For example, the controller 106 may be coupled to ormay otherwise utilize one or more remote computer systems and/or othersystems, such as the electronic system 118, the infotainment system 148of the vehicle 100, and/or one or more other systems of the vehicle 100.

In the depicted embodiment, the computer system of the controller 106includes a processor 172, a memory 174, an interface 176, a storagedevice 178, and a bus 180. The processor 172 performs the computationand control functions of the controller 106, and may comprise any typeof processor or multiple processors, single integrated circuits such asa microprocessor, or any suitable number of integrated circuit devicesand/or circuit boards working in cooperation to accomplish the functionsof a processing unit. During operation, the processor 172 executes oneor more programs 182 contained within the memory 174 and, as such,controls the general operation of the controller 106 and the computersystem of the controller 106, generally in executing the processesdescribed herein, such as the processes described with reference to thefigures below.

The memory 174 can be any type of suitable memory. For example, thememory 174 may include various types of dynamic random access memory(DRAM) such as SDRAM, the various types of static RAM (SRAM), and thevarious types of non-volatile memory (PROM, EPROM, and flash). Incertain examples, the memory 174 is located on and/or co-located on thesame computer chip as the processor 172. In the depicted embodiment, thememory 174 stores the above-referenced program 182 along with one ormore stored values 184.

The bus 180 serves to transmit programs, data, status and otherinformation or signals between the various components of the computersystem of the controller 106. The interface 176 allows communication tothe computer system of the controller 106, for example from a systemdriver and/or another computer system, and can be implemented using anysuitable method and apparatus. In one embodiment, the interface 176obtains the various data from the sensors of the sensor array 104. Theinterface 176 can include one or more network interfaces to communicatewith other systems or components. The interface 176 may also include oneor more network interfaces to communicate with technicians, and/or oneor more storage interfaces to connect to storage apparatuses, such asthe storage device 178.

The storage device 178 can be any suitable type of storage apparatus,including direct access storage devices such as hard disk drives, flashsystems, floppy disk drives, an SD card and optical disk drives. In oneexemplary embodiment, the storage device 178 comprises a program productfrom which memory 174 can receive a program 182 that executes one ormore embodiments of one or more processes of the present disclosure, aswill be described further below, particularly with respect to FIGS. 4,5, 7 and 10. In another exemplary embodiment, the program product may bedirectly stored in and/or otherwise accessed by the memory 174 and/or adisk (e.g., disk 186), such as that referenced below. The storage device178 may also store video data 204 captured by the one or more cameras103. The video data 204 may be stored when the remote communicationsmodule 202 is not able to connect to a cellular network 1000 or even ifthe video data has also been streamed or otherwise sent to a remoteserver 900 or remote user device 800.

The bus 180 can be any suitable physical or logical means of connectingcomputer systems and components. This includes, but is not limited to,direct hard-wired connections, fiber optics, infrared and wireless bustechnologies. During operation, the program 182 is stored in the memory174 and executed by the processor 172.

It will be appreciated that while this exemplary embodiment is describedin the context of a fully functioning computer system, those skilled inthe art will recognize that the mechanisms of the present disclosure arecapable of being distributed as a program product with one or more typesof non-transitory computer-readable signal bearing media used to storethe program and the instructions thereof and carry out the distributionthereof, such as a non-transitory computer readable medium bearing theprogram and containing computer instructions stored therein for causinga computer processor (such as the processor 172) to perform and executethe program. Such a program product may take a variety of forms, and thepresent disclosure applies equally regardless of the particular type ofcomputer-readable signal bearing media used to carry out thedistribution. Examples of signal bearing media include: recordable mediasuch as floppy disks, hard drives, memory cards and optical disks, andtransmission media such as digital and analog communication links. Itwill be appreciated that cloud-based storage and/or other techniques mayalso be utilized in certain embodiments. It will similarly beappreciated that the computer system of the controller 106 may alsootherwise differ from the embodiment depicted in FIG. 1, for example inthat the computer system of the controller 106 may be coupled to or mayotherwise utilize one or more remote computer systems and/or othersystems.

The display system 108 is coupled to the controller 106. The displaysystem 108 comprises a display screen 191. In one embodiment, thedisplay screen 191 provides a visual display of photographic and/orrecorded video images and data from the one or more cameras 103, viainstructions provided by the processor 172, for viewing by a user withinthe vehicle 100.

FIG. 2 shows a network 400 of the vehicle 100, specifically the controlsystem 102 of the vehicle 100, a remote server 900 including a servercontrol system 902, and one or more user devices 800 including one ormore device control systems 802. The vehicle control system 102, theremoter server 900 and the device control systems are in communicationover a cellular network 1000. The network 400 allows video to bestreamed from the vehicle 100 to either the remote server 900 or theuser device 800. The network also allows control commands to be sentfrom the remote server 900 or the user device 800 to the vehicle 100relating to operation of the one or more cameras 103 and/ortransmission, e.g. streaming, of video captured by the one or morecameras 103 and/or other vehicle control commands, as will be describedin further detail below.

The remote server 900 may be staffed by one or more (human) operators(not shown) to allow audio and optionally also video communications witha driver of the vehicle (or a passenger). Such conferencing between theremote operator and the driver may be required to allow the driver tospeak to a live operator at any time of day. The vehicle control systemcomprises one or more conferencing input devices 622, 624 or other inputdevices to initiate a remote conference between an operator at theremote server 900 and a driver or passenger of the vehicle 100. Theoperator can set up turn-by-turn directions, look up the address of apoint of interest, or make changes to an account. The vehicle driver canalso request a live diagnostic checkup, in which case the operator willpull information from the vehicle control system 102. Such conferencingalso allows connection with an operator who is trained to deal withemergencies. The operator is able to contact the police, firedepartment, or request medical assistance in the case of an emergency.As will be explained below, the vehicle control system 102 may comprisetwo input buttons or other input devices 622, 624; one for conferencingwith an operator for non-emergency situations and one for emergencies.It should be understood that the input buttons 622, 624 may bemechanical, electromechanical or graphics buttons on a display screen(such as vehicle display screen 191).

The user device 800 may be a mobile telephone, a tablet device, adesktop computer, a laptop or any other computing and communicationsdevice.

FIG. 3 is a schematic view of control systems 102, 802, 902 configuredfor carrying out the network 400 and the various processes describedfurther below. The vehicle control system 102 has been described indetail in the foregoing. Also shown is a server control system 902 and auser device control system 802. These control systems 102, 802, 902 arein communication with one another via a cellular network 1000. Invarious embodiments, the server control system 902 and the user devicecontrol system 802 are both included. In alternative embodiments, justone of the server control system 902 and the user device control system802 is included in the network 400.

The server control system 902 comprises a computer system including acontroller 910, a remote communications module 904, a memory 908, a datastorage device 912 and an operator interface 906. In the depictedembodiment, the computer system of the controller 910 includes aprocessor (not shown), the memory 908, the storage device 912, and a bus(not shown). The processor performs the computation and controlfunctions of the controller 910, and may comprise any type of processoror multiple processors, single integrated circuits such as amicroprocessor, or any suitable number of integrated circuit devicesand/or circuit boards working in cooperation to accomplish the functionsof a processing unit. During operation, the processor or controller 910executes one or more programs contained within the memory 908 and, assuch, controls the general operation of the controller 910 and thecomputer system of the controller 910, generally in executing theprocesses described herein, such as the processes described withreference to the figures below. The one or more programs may include theapp described further below.

The memory 908 can be any type of suitable memory. For example, thememory 908 may include various types of dynamic random access memory(DRAM) such as SDRAM, the various types of static RAM (SRAM), and thevarious types of non-volatile memory (PROM, EPROM, and flash). Incertain examples, the memory 908 is located on and/or co-located on thesame computer chip as the processor. In the depicted embodiment, thememory 908 stores one or more programs for executing at least part ofthe processes described herein, particularly the server controlledaspects of the processes.

The bus serves to transmit programs, data, status and other informationor signals between the various components of the computer system of thecontroller 910.

The storage device 912 can be any suitable type of storage apparatus,including direct access storage devices such as hard disk drives, flashsystems, floppy disk drives and optical disk drives. In one exemplaryembodiment, the storage device 912 comprises a program product fromwhich memory 908 can receive a program that executes one or moreembodiments of one or more processes of the present disclosure, as willbe described further below. In another exemplary embodiment, the programproduct may be directly stored in and/or otherwise accessed by thememory 908. The storage device 912 may also store video data 920captured by the one or more cameras 103 of the vehicle 100 and sent tothe remote server 900 via the cellular network 1000.

The bus can be any suitable physical or logical means of connectingcomputer systems and components. This includes, but is not limited to,direct hard-wired connections, fiber optics, infrared and wireless bustechnologies. During operation, one or more programs are stored in thememory 908 and executed by the processor.

The server control system 902 also comprises an operator interface 906,which may include an input unit 916. The input unit 916 may includeaudio, keyboard, mouse, touchscreen and other input capabilities. Theoperator interface 906 may include audio, image and video outputcapabilities. The operator interface 906 may include a display unit 914and speakers for outputting image, video and audio data to an operator.

The controller 910 is operable with the remote communications module 904to receive video data sent over the cellular network 1000. In variousembodiments, the controller 910 is operable to receive sound datatransmitted over the cellular network 1000. The remote communicationsmodule 904 may comprise a control unit and a transceiver fortransmitting and receiving data over the cellular network 1000. Theremote communications module 904 may operate through third generation(3G) or fourth generation (4G) standards or any other telecommunicationsstandard. In various embodiments, the controller 910 is configured tostore the video data (and optionally the sound data) received over thecellular network 1000 in the data storage device 912. In variousembodiments, the controller 910 is configured to output video data (andsound data) received from the vehicle 100 over the cellular networkthrough the operator interface 906 to allow an operator to view outsidesurroundings of the vehicle 100 and/or the interior of the vehicle 100.In various embodiments, sound data is stored in the data storage device912 and/or is output through the operator interface so that the videodata and the sound data can be synchronously replayed. In this way, itis possible to ascertain further information on any event of interest.

The user device control system 802 comprises a computer system includinga controller 810, a remote communications module 804, a memory 808, datastorage device 812 and a user interface 806. In the depicted embodiment,the computer system of the controller 810 includes a processor (notshown), the memory 808, the storage device 812, and a bus (not shown).The processor performs the computation and control functions of thecontroller 810, and may comprise any type of processor or multipleprocessors, single integrated circuits such as a microprocessor, or anysuitable number of integrated circuit devices and/or circuit boardsworking in cooperation to accomplish the functions of a processing unit.During operation, the processor 810 executes one or more programscontained within the memory 908 and, as such, controls the generaloperation of the controller 810 and the computer system of thecontroller 810, generally in executing the processes described herein,such as the processes described with reference to the figures below. Theone or more program may include the app described in further detailbelow.

The memory 808 can be any type of suitable memory. For example, thememory 808 may include various types of dynamic random access memory(DRAM) such as SDRAM, the various types of static RAM (SRAM), and thevarious types of non-volatile memory (PROM, EPROM, and flash). Incertain examples, the memory 808 is located on and/or co-located on thesame computer chip as the processor. In the depicted embodiment, thememory 808 stores one or more programs for executing at least part ofthe processes described herein, particularly the user device controlledaspects of the processes.

The bus serves to transmit programs, data, status and other informationor signals between the various components of the computer system of thecontroller 810.

The storage device 812 can be any suitable type of storage apparatus,including direct access storage devices such as hard disk drives, flashsystems, floppy disk drives and optical disk drives. In one exemplaryembodiment, the storage device 812 comprises a program product fromwhich memory 808 can receive a program that executes one or moreembodiments of one or more processes of the present disclosure, as willbe described further below. In another exemplary embodiment, the programproduct may be directly stored in and/or otherwise accessed by thememory 808. The storage device 812 may also store video data 820captured by the one or more cameras 103 of the vehicle 100 and sent tothe user device 800 over the cellular network 1000, either from thevehicle 100 or from the remote server 900.

The bus can be any suitable physical or logical means of connectingcomputer systems and components. This includes, but is not limited to,direct hard-wired connections, fiber optics, infrared and wireless bustechnologies. During operation, one or more programs are stored in thememory 908 and executed by the processor.

The user device control system 802 also comprises a user interface 806,which may include an input unit 916. The input unit 916 may includeaudio, keyboard, mouse, touchscreen and other input capabilities. Theoperator interface 906 may include audio, image and video outputcapabilities. The user interface 806 may include a display unit 814,optionally in the form of a touchscreen display, and speakers foroutputting image, video and audio data to a user. The user interface 806allows a user to view video and/or images captured by the one or morecameras 103. In various embodiments, the user interface 806 and thecontroller 910 allows a user to send control commands to controller 106of the vehicle control system 102 to control operation of the one ormore cameras 103 and sending of video data from the vehicle 100. Thecontrol commands may be sent via the remote server 900 or more directlyto the vehicle 100.

The controller 810 is operable with the remote communications module 804to receive video and image data sent over the cellular network 1000,which may be received from the remote server 900 or from the vehicle100. In some embodiments, the controller 810 is operable with the remotecommunications module 804 to receive sound data from the vehicle 100 inaddition to the video data. The sound data is captured by the one ormore microphones 289. In various embodiments, video data (and optionallyalso sound data) from the vehicle 100 must first be transmitted to theremote server 900 from which video data is then transmitted to the userdevice 800. The remote communications module 804 may comprise a controlunit and a transceiver for transmitting and receiving data over thecellular network 1000. The remote communications module 804 may operatethrough third generation (3G) or fourth generation (4G) standards or anyother telecommunications standard. In various embodiments, thecontroller 810 is configured to store the video and image data (andoptionally sound data) received over the cellular network 1000 in thedata storage device 812. In various embodiments, the controller 810 isconfigured to output video data (and optionally sound data) receivedfrom the vehicle 100 over the cellular network 1000 through the userinterface 806 to allow a user to view (and optionally hear) outsidesurroundings of the vehicle 100 and/or the interior of the vehicle 100.

Remote Video Access and Control

In various embodiments, the one or more cameras 103 are configured tocapture video data of the outside surroundings of the vehicle 100 and/orthe interior of the vehicle 100. Further, the one or more microphones289 are configured to capture sound data of the outside surroundings ofthe vehicle 100 and/or the interior of the vehicle 100. The controller106 is configured to send, e.g. stream, the video data and any sounddata over a cellular or other telecommunications network 1000 via aremote communications module 202. The controller 910 of the server 900and/or the controller 810 of the user device 800 is/are configured toreceive the video data and any sound data via the respective remotecommunications module 804, 904. The controller 910 of the server 900 mayreceive the video data and any sound data via an internet connectionincluded in the remote communications module 904. The internetconnection may be wired broadband access such as DSL, wireless broadbandsuch as satellite or mobile broadband through the cellular network 1000.The controller 810 of the user device 800 may receive the video data andany sound data via an internet connection included in the remotecommunications module 804. The internet connection may be wiredbroadband access such as DSL, wireless broadband such as satellite ormobile broadband through the cellular network 1000. In an embodiment,the controller 810 of the user device is a mobile telephone and is ableto receive the video data and any sound data through wireless internetconnection. The user device 800 may be configured to connect to theremote server 900 (which are themselves remote from each other) in orderto receive the video data and any sound data from the remote server 900.In this instance, the remote server 900 is configured to receive thevideo data and any sound data from the vehicle 100 first and to send thevideo data to the user device 800.

In various embodiments, the one or more cameras 103 include a pluralityof cameras 103 able to capture video data from respective sides of thevehicle 100. The controller 106 may be configured to stitch the videodata from respective cameras 103 together in order to provide panoramicvideo data of most of the outside surroundings of the vehicle or even360° panoramic surround video data of the vehicle 100. In anotherembodiment, the controller 106 may be configured to capture and send thevideo data in the form of a mosaic of video data, with each video of themosaic corresponding to a view from a respective camera 103. Thecontroller 106 is configured to send or stream the panoramic or mosaicvideo data from the vehicle 100 over the cellular network 1000 via theremote communications module 202. In an alternative embodiment, thecontroller 106 is configured to send or stream the video data from theplurality of cameras 103 over the cellular network 1000 via the remotecommunications module and stitching of the video data to createpanoramic video data is carried out by a remote controller, such as theserver controller 910 or the user device controller 810. The panoramicvideo data may include sound data as mentioned above.

In an embodiment, the display unit 914 of the server control system 902is configured to display the video data, thereby enabling a serveroperator to view the interior and/or the outside surroundings of thevehicle 100. Further, the server control system 902 may comprise one ormore speakers (not shown) to play the sound data. In an additional oralternative embodiment, the display unit 814 of the user device controlsystem 802 is configured to display the video data, thereby enabling auser to view the interior and/or the outside surroundings of the vehicle100. Further, the user device 800, via the user device control system802 and speakers (not shown) is configured to play the sound data. Thevideo data and any sound data may be streamed from the vehicle 100 suchthat the server operator and/or the user is viewing real-time video andcan hear any sound data in real time.

In various embodiments, the server control system 902 and/or the userdevice control system 802 is/are configured to send one or more controlcommands to the vehicle control system 102. The control commands may betransmitted at least partly over the cellular network. The one or morecontrol commands may be transmitted via the remote communications module804, 904. In particular, the control commands are able to control theone or more cameras 103 or the video data captured thereby in oneembodiment.

In various embodiments, a graphical user interface 600 as shown in FIG.6 may be provided for display on a display unit 814, 914 of the remoteserver 900 or the user device 800. The graphical user interface 600 isconfigured to display selection graphics 602 to 618 to allow a user oroperator to select an appropriate control.

Various such control commands are discussed in the following inaccordance with various exemplary embodiments. The various controlcommands may be provided alone or in any combination.

Wake-Up and Standby Commands

One exemplary control command is a wake-up or video ON command. Thewake-up command, when received and processed by the vehicle controlsystem 102, is configured to activate the one or more cameras 103 and toactivate sending or streaming of captured video data over the cellularnetwork 1000 to the server control system 902 or the user device controlsystem 802. The wake-up command could be initiated by a user wanting tocheck on the security of the vehicle, particularly in instances wherethe vehicle has been parked in a potentially unsecure location such as apublic parking lot. Other situations can be envisaged when a remoteactivated check of a vehicle's surroundings and/or interior would beuseful. The wake-up command, when received and processed by the vehiclecontrol system 102, may also be configured to activate the one or moremicrophones 289 such that sound and video data are sent or streamed overthe cellular network 1000.

In particular, a user may operate the input unit 816 of the user device800 to select the wake-up or video ON command. This may involvelaunching of an application on the user device 800, which is run by thecontroller 810. In an embodiment, the wake-up command may be selectedwithin an app being run on the user device 800. The controller 810,responsive to the user input through the input unit 816, is configuredto send the wake-up command using the remote communications module 804.The user input may be carried out by operation of a touch-screen displayembodying the input unit 816. Alternatively, an operator may operate theinput unit 916 of the server 900 to select a wake-up command. This mayinvolve launching of an application on the server 900, which is run bythe controller 910. In an embodiment, the wake-up command may beselected within an app being run on the server 900. The controller 910,responsive to the user input through the input unit 916, is configuredto send the wake-up command using the remote communications module 904.The user input may be carried out by operation of a touch-screen displayembodying the input unit 916.

The vehicle control system 102 is configured to be responsive to thewake-up command, received via the remote communications module 202, toactivate the one or more cameras 103 and optionally the one or moremicrophones 289 and to activate sending of the video data and sound datacaptured thereby. The user device control system 802 and/or the remoteserver control system 902 is/are configured to receive the video datavia the remote communications module 804, 904. The controller 810 of theuser device 800 may be operable to display the video data on the displayunit 814 and the controller 910 of the service may be operable todisplay the video data on the display unit 914. In this way, the user isable to view video data remotely, which is streamed from the vehicle100. Further, the vehicle 100 is not required to continuously operatethe one or more cameras 103 and to send the captured video data. This isable to save data costs as well as vehicle power consumption.

In various embodiments, the server control system 902 and/or the userdevice control system 802 is/are configured, in conjunction with theinput unit 816, 916, so that a user or operator is able to select toturn video OFF. The server control system and/or the user device controlsystem 802 is/are configured to responsively transmit a standby commandto the vehicle control system 102. The vehicle control system 102 isconfigured to be responsive to the standby command to place the one ormore cameras 103 (and optionally also the one or more microphones 289)in a standby mode so that video data is not captured and/or the vehiclecontrol system 102 is configured to disable sending of video data. Inthis way, cellular data is not transmitted when it is not requested by auser to save on data costs and power consumption. Alternatively, theuser device control system 802 and/or the server control system 902 isconfigured to periodically transmit the wake-up command or a similarcommand indicating that video data should be streamed by the vehiclecontrol system 102 as long as a video selection on the user device 800and/or the remote server 900 persists. When a selection is made to turnOFF the video, the user device control system 802 and/or the servercontrol system 902 is/are configured to discontinue transmitting thewake-up or similar command. The vehicle control system 102 is configuredto respond to the absence of the wake-up or similar command by enteringthe one or more cameras 103 into a standby mode or disabling sending ofvideo data.

FIG. 4 shows an exemplary flowchart of a process of selectivelyactivating one or more cameras of a vehicle 100 from a remote site.

In step 500, capturing of the video data by the one or more cameras 103and sending of the captured video data over the cellular network 1000 isin a standby state. It may be that the one or more cameras 103 are in astandby state by video data not being captured by the one or morecameras 103 (i.e. the cameras 103 themselves are on standby) or that thevideo data is being captured by the one or more cameras 103, but sendingof the video data over the cellular network or the telecommunicationsnetwork 1000 is currently disabled.

In step 502, a user or operator initiates a wake-up or video ON commandthat is transmitted from the user device 800 or from the remote server900. The user or operator may initiate such a wake-up command by aselection within a graphical user interface of an application running onthe controller 810, 910 (as described further with reference to FIG. 6)or by launching the application itself. The selection within theapplication may correspond to a graphical button, icon, etc., where thebutton, icon, etc. represents a selection of viewing of one or morevehicle cameras 103. The selection within the application or launchingof the application may be carried out by touchscreen selection, mouseinput, keyboard input, etc. The application or app referred to hereinmay correspond to a computer program being executed by a processor of atleast one of the controllers 106, 810, 910.

In step 504, the one or more vehicle cameras 103 are, in response to thewake up command, operated to capture video data. In various embodiments,the one or more microphones 289 are, in response to the wake up command,operated to capture sound data. In certain embodiments, a plurality ofvideo cameras 103 are operated to allow generation of panoramic videodata. The video data from each camera 103 may be stitched before orafter the video data transmission step 506. Alternatively, the videodata may remain unstitched to allow video data from each camera 103 tobe separately viewed in various embodiments.

In step 506, the captured video data and any sound data are transmittedto the remote server 900 and/or the user device 800. In embodiments, thevideo data and any sound date are transmitted at least partly over acellular network 1000. The video data and any sound data are receivedand processed at the remote server 900 and/or the user device 800. Theprocessing may involve decompression of the video data and/or stitchingof video data from different vehicle cameras 103 to provide a panoramicview. The processing may involve integrating a sound channel includingthe sound data into the video data to allow for synchronous sound andvideo replay.

In step 508, the received video data is displayed at the remote server900 or the user device 800. The remote server 900 and the user device800 include respective display units 814, 914 that are operable with therespective controllers 810, 910 to display the video data. Further, theremote server 900 or the user device 800 may be operable to reply thesound data through one or more speakers (not shown).

In step 510, the user or operator selects to turn the video off byoperation of the input unit 816, 916. Once the selection to turn off thevideo has been made, receipt, i.e. streaming, of the video data and anysound data is stopped and display of the video data is also stopped. Theuser may select to turn the video off by operating a touchscreen, mouse,etc. embodying the input unit. In particular, a graphic or iconcorresponding to video OFF may be selected, as described further belowwith reference to FIG. 6.

In step 512, after the user or operator selects to turn OFF the video,capturing of video data by the one or more cameras 103 and sending ofthe captured video data from the vehicle 100 is returned to standby. Inthis way, cellular data use is not extended beyond the user'srequirements. The server 900 or the user device 800 may return a standbycommand after the video OFF selection is made. The standby command istransmitted from the user device 800 or the remote server 900 to thevehicle 100 at least partly over the cellular or telecommunicationsnetwork 1000. Alternatively, the absence of a command from the userdevice 800 or the server 900 may be interpreted by the vehicle controlsystem 102 as a standby command.

Video Responsive Commands

Another exemplary control command that may be remotely transmitted tothe vehicle control system 102 is a video responsive control command,which is responsive to the content of the video. For example, a videoenhancement control command may be sent. In various embodiments, thevideo responsive command is selected from a camera wash command, acamera articulation command, an illumination command, a camera changecommand, camera low light enhancement command, and a sound activationcommand. These commands are all responsive to the content of video databeing viewed. For example, if the video being viewed is unclear, acamera wash may be required. If the video being viewed is not showinginformation of relevance, a change of camera or camera articulation maybe required. If the video being viewed shows vandalism or theft inprogress, the illumination and sound activation may be of assistance. Ifthe video being viewed is too dark, illumination may be required. Theillumination command and the sound activation command may also have atheft or vandalism deterrent effect and thus can be considered securityrelated alarms. Similarly, if the video being viewed is too dark, acamera low light enhancement command may be instructed. The low lightenhancement command may adjust settings of the camera, such as framerate, exposure time or frequency sensitivity, to enhance captured videodata depending upon the lighting conditions.

The server control system 902 and/or the user device control system 802may be configured to send such a video responsive control command atleast partly over the cellular or telecommunications network 1000 viathe respective remote communications module 804, 904. The vehiclecontrol system 102 is configured to receive the video responsive controlcommand via the remote communications module 202. The vehicle controller106 is configured to be responsive to the control command by running aprocess to implement the command.

For example, in one embodiment the vehicle controller 106 is configuredto respond to a camera wash command by issuing a corresponding controlcommand to the one or more camera washers or wipers 280 to clean theassociated camera 103. In various embodiments, the camera wash commandtransmitted from the remote server 900 and/or the user device 800 may becamera specific, i.e. designates a subset (e.g. just one) of a set ofcameras 103 of the vehicle 100 or it may be non-specific. The vehiclecontroller 106 may be configured to respond to the specific wash commandby initiating a wash or wipe cycle of the washers or wipers 280 of thesubset of cameras 103. The vehicle controller 106 may be configured torespond to the non-specific wash command by initiating a wash or wipecycle of the washers or wipers 280 of all of the cameras 103.

In various embodiments, the vehicle controller 106 is configured torespond to a camera articulation command by issuing a correspondingcommand to the one or more camera articulators 282 to change the fieldof the view of the associated camera 103. Subsequently, video datacaptured by the camera 103 will have a different view than beforearticulation of the camera 103. The vehicle control system 102 isconfigured to send the video data from the new camera view, which can bereceived and displayed by the server control system 902 and/or the userdevice control system 802. In this way, a user or operator is able tocontrol, from a remote location, a view of the camera 103. In variousembodiments, the camera articulation command transmitted from the remoteserver 900 and/or the user device 800 is camera specific, i.e.designates a subset (e.g. just one) of a set of cameras 103 of thevehicle 100.

In various embodiments, the vehicle controller 106 is configured torespond to an illumination command to illuminate a field of view of theone or more cameras 103. In various embodiments, the vehicle controller106 is configured to be responsive to the illumination command byswitching ON the one or more illumination devices 284, 286. The vehiclecontroller 106 may be configured to switch OFF the one or moreillumination devices 284, 286 after a predetermined amount of time orthe switch OFF of the illumination devices may be embodied in anillumination OFF command from the remote server and/or the remote userdevice 800 to which the vehicle controller 106 is responsive. Theillumination command transmitted from the remote server 900 and/or theuser device 800 may be illumination device specific, i.e. designates asubset (e.g. just one) of a set of illumination device 284, 286 of thevehicle 100 or it may be non-specific such that all illumination devices284, 286 are designated. For example, a user or operator may be viewingvideo data captured by a subset (e.g. just one) of a plurality ofcameras 103. The user or operator may note that the view from the subsetof cameras 103 is insufficiently bright and select an illumination ONinput through the input unit 816, 916. In embodiments, the servercontrol system 902 and/or the user device control system 802 is/areconfigured to respond to the user input by transmitting an illuminationdevice specific illumination control command to light the view of thesubset of cameras 103. The vehicle control system, particularly thevehicle controller 106, is configured to respond to the specific ornon-specific illumination command by switching ON the correspondingillumination device or devices 284, 286.

In various embodiments, the vehicle controller 106 is configured torespond to a low light enhancement command by adjusting one or moresettings of the one or more cameras 103 to enhance video data as captureby the one or more cameras 103 in low light conditions. For example,settings of the camera, such as frame rate, exposure time or frequencysensitivity may be adjusted. The frame rate may be lowered, the exposuretime increased or the frequency sensitivity shifted to the infraredregion to provide enhanced video data under low lighting conditions. Theuser or operator may note that the view from the subset of cameras 103is insufficiently bright and select a low light enhancement commandthrough the input unit 816, 916. In embodiments, the server controlsystem 902 and/or the user device control system 802 is/are configuredto respond to the user input by transmitting a low light enhancementcontrol command to capture video data using camera settings optimizedfor low lighting conditions. The vehicle control system, particularlythe vehicle controller 106, is configured to respond to the low lightenhancement command by adjusting one or more settings of the one or morecameras 103.

In various embodiments, the vehicle 100 comprises a plurality of cameras103, each providing a different camera view. The vehicle controller 106is configured to respond to a camera change command by switching fromone or more currently active cameras 103 to one or more other cameras103 to change the view of video data being transmitted from the vehicle100. This can be implemented by the vehicle controller 106 beingconfigured to switch OFF one or more of the currently active cameras 103and switching ON one or more other cameras 103. Additionally oralternatively, the vehicle controller 106 can be configured to capturevideo data with each of the plurality of cameras 103 and to switchsending video data from one or more cameras 103 to one or more othercameras 103 over the cellular or telecommunications network 1000 via theremote communications module 202. A user or operator is able to select acamera view using the respective input unit 816, 916. The server controlsystem 902 and/or the user device control system is/are configured torespond to the user input by transmitting the camera change commandbased on the user or operator selection of camera view. The user oroperator may, for example, select a left side, right side, front or rearview (or combinations thereof). The vehicle control system 102 isconfigured to respond to the camera change command by returning thevideo data corresponding to user's camera view selection or theoperator's camera view selection. The controller 810, 910 of the server900 or the user device 800 is/are configured to render the returnedvideo data to the display unit 814, 914 to show a change of viewaccording to the selection of the user or the operator.

In various embodiments, the vehicle controller 106 is configured torespond to a sound activation command by operating the one or more sounddevices or speakers 288 so as to sound the horn, an alarm or speechwarning, or the like, to warn third parties away from any securitybreach of the vehicle 100. The vehicle controller 106 may be configuredto sound the warning through the one or more speakers 288 for apredetermined amount of time. Additionally or alternatively, the servercontrol system 902 and/or the user device control system 802 is/areconfigured to transmit a sound activation OFF command to which thevehicle control system 102 is responsive to switch OFF the soundwarning. The server control system 902 and/or the user device controlsystem is/are configured to generate the sound activation OFF commandresponsive to a corresponding selection by a user or operator throughthe input unit 816, 916.

FIG. 5 shows a flowchart illustrating steps of an exemplary process oftransmitting video responsive control commands from a remote user device800 or a remote server 900 to a vehicle 100.

In step 520, video data, captured by one or more cameras 103 (andoptionally sound data captured by the one or more microphone 289), istransmitted from a vehicle 100 at least partly over the cellular ortelecommunications network 1000.

In step 522, the video data is received at the remote server 900 and/orthe remote user device 800 and displayed on the respective display unit814, 914. The user or operator is able to view the interior and/or thesurroundings of the vehicle on the display unit 814, 914. In variousembodiments, the user and/or operator may note a video content issuesuch as a video quality issue or a problem with the camera view and/or asecurity issue by viewing the video data.

In step 524, the user and/or operator is able to take action in responseto the video content issue and/or the security issue. In particular, theuser and/or operator is able to operate the input unit 816, 916, e.g. byway of mouse click, keyboard entry, touchscreen selection, etc., toselect a video responsive control action. The video responsive controlaction may comprise a video quality enhancement action (such as camerawash action, illumination action, and/or sound action) or a video viewchange action (such as camera articulation action and/or active camerachange action) and/or a security response action (such as a soundactivation action and/or an illumination action).

In step 526, the remote server control system 902 and/or the user devicecontrol system 802 responds to the user or operator selection totransmit a corresponding video responsive control command.

According to step 528, in various embodiments, the vehicle controlsystem 102 changes one or more parameters of the video data or affectingthe video data sent from the vehicle 100 to the remote server 900 and/orthe remote user device 800. For example, the video data may come fromone or more different cameras 103 as a result of a camera changecommand. In another example, the video data may be captured from one ormore cameras that have been articulated as a result of an articulationcommand. In another example, the video data may come from one or morecameras 103 that have been washed or wiped as a result of a camera washcommand. In another example, the video data may from one or more cameras103 for which a field of view has been illuminated as a result of anillumination command.

According to step 528, in various embodiments, the one or more vehiclespeakers are sounded and/or one or more illumination devices 284, 286,are switched ON. These steps can act as a vandalism and theft deterrent.

In step 530, video data is sent from the vehicle control system 102 tothe remote server control system 902 and/or the user device controlsystem 802. The remote server control system 902 and/or the user devicecontrol system 802 renders and displays the video data on the displayunit 814, 914, which can be viewed by a user or operator. Depending onthe command, the user or operator may be able to see a betterilluminated video, a video enhanced for low lighting conditions, a videotaken from a different camera or a different camera angle of the samecamera or the quality of the video may have been enhanced by camerawashing or wiping. The user or operator is thus able to remotely controlthe one or more cameras 103, video data from the one or more cameras 103or other aspects affecting the usefulness of the video data. Since thesechanges are made at the vehicle end, data costs can be saved. Further,the user or operator may be able to view in the video an effectivedeterrent of sound activation or illumination device activation.

Amount of Data Command

Another exemplary control command that can be sent from the remoteserver 900 and/or the remote user device 800 to the vehicle 100 couldrelate to the amount of video data being sent. In particular, the useror operator may wish to reduce or increase the amount of video databeing sent from a vehicle 100 and received by the remote user device 800and/or the remote server 900. Such control capability is facilitated byan amount of data control command as described in the following.

In various embodiments, the user device control system 802 and/or theserver control system 902 is/are configured to allow a user or operatorto select, through the input unit 816, 916, one or more parametersrelating to an amount of video data received. The input 816, 916 may beconfigured so that a user can select to increase or decrease the amountof video data received. The input unit 816, 916 is configured to allow auser to select any one or more of data resolution, data compression,color or grayscale, frames per second, number of active cameras 103,etc. in order to allow control of increase or decrease of amount ofdata.

In various embodiments, the controller 810, 910 of the user device 800or the remote server 900 is configured to interpret the user or operatorinput selection and to generate an amount of data control command. Thecontroller 810, 910 is configured to operate in conjunction with theremote communications module 804, 904 to transmit the amount of datacontrol command over the internet and/or the cellular ortelecommunications network 1000 to the vehicle control system 102. Thevehicle controller 106 is configured to operate with the remotetelecommunications module 202 to receive the amount of data controlcommand from the cellular or telecommunications network 1000. Thecontroller 106 is configured to interpret the amount of data controlcommand and to adjust one or more parameters of video data that has beencaptured by the one or more cameras 103. Depending on the amount of datacontrol command, the controller 106 is configured to change the amountof data transmitted over the cellular or telecommunications network 1000in order to increase or decrease data usage depending upon userpreferences as defined by the amount of data control command.

There are a number of exemplary parameters that may be adjusted tochange data usage in transmitting video data from a vehicle 100. Invarious embodiments, the controller 106 is configured to change theframe rate or frequency of transmission of the video data to change theamount of data being transmitted. In various embodiments, the controller106 is configured to change the resolution of the video data captured bythe one or more cameras to change data usage. Thus, a camera setting maybe changed by the controller 106 to change the resolution of the videodata being captured. In various embodiments, the controller 106 isconfigured to change the data rate of the video data transmitted fromthe vehicle 100. That is, the controller 106 is configured to compressthe video data to change the data rate. To do so, settings of an encoderused for data compression are changed. The data rate may be changed fromtens of kilobytes per second to at least one and even tens of megabytesper second. In various embodiments, the video data may be captured bythe one or more cameras 103 or transmitted, under the instruction of thecontroller 106, in grayscale or color to change the amount of data.Further, the level of grayscale and color may be changed to change theamount of data being transmitted from the vehicle.

In various embodiments, the amount of data control command is able toadjust a size of a field of view of video data transmitted from thevehicle 100 when received, interpreted and executed by the vehiclecontroller 106. For example, the data control command may define anumber of cameras 103 from which video data is transmitted. In oneembodiment, the vehicle 100 may include sufficient cameras 103 toprovide substantially 360° surround video data. Such surround video datamay be stitched together or be provided by a mosaic of separated videos.The amount of data control command is able to select a subset (e.g. justone) of the cameras 103 to limit the view size, thereby limiting thedata usage requirement. That is, the amount of data control command maychange from 360° surround video data from a single camera and viceversa. Further, gradations between maximum surround and video data froma single camera 103 are possible, such as video data from two cameras103 where the vehicle 100 includes four or more cameras 103. In anotherpossibility, the amount of data control command may define a honing ofthe video data. That is, partial views included in the video data, suchas video data from a single camera, may be defined. The amount of datacontrol command may be able to define the size of the partial view whenreceived and executed by the controller 106. According to these variousembodiments, the amount of data transmitted by the vehicle 100 is ableto be adjusted through an amount of data control command generated as aresult of a user or operator selection at a remote server 900 or aremote user device 800.

The amount of data control command allows for control of the amount ofdata transmitted from a vehicle 100, thereby potentially reducingupstream data costs, as well as potentially reducing downstream datacosts at the remote server 900 or the remote user device 800. The datacontrol command also allows a user or operator to increase the amount ofdata, and thus the quality of the video, on command. Such functionalitymay be useful if a user or operator notes a security threat as a resultof viewing the video data.

In various embodiments, a process for controlling the amount of dataused by the network 400 of vehicle 100 and remote server 900 and/or theremote user device 800 is also shown in FIG. 5.

In step 520 video data is streamed or transmitted in real time from thevehicle 100 to the remote server 900 and/or the remote user device 800at least partly over the cellular network 1000.

In step 522, the video data received at the remote server 900 and/or theremote user device 800 is displayed on the display unit 814, 914. Thevideo data may be substantially 360° surround video data or video datafrom one, some or all of the front, rear, left side, ride side andinterior of the vehicle 100. In the case of surround video data, such assubstantially 360° surround video data, the user or operator is able toscroll through the stitched surround view or through respective views ofa mosaic of videos by operating the input unit 816, 916.

In step 524, the operator or user selects to change the data usage usingthe input unit 816, 916. This may be a qualitative selection such aslow, medium or high. For a qualitative selection, the controller 810,910 may access predetermined settings from the memory 808, 908 definingan amount of data transmitted from the vehicle 100. The operator or usermay additionally or alternatively be able to select specific data usageparameters for adjustment such as frame rate, view size, grayscale orcolor, data rate, etc. as described above. The operator or user may makeuse of a scroll bar, data usage values, buttons, icons, arrows, etc. inorder to make the data usage selection.

In step 526, one or more data usage parameters or settings areformulated into one or more data amount control commands by thecontroller 810, 910. The one or more data amount control commands aretransmitted from the remote user device 800 and/or the remote server 900by operation of the controller 810, 910 in conjunction with the remotecommunications module 804, 904. The one or more amount of data controlcommands are received by the vehicle control system 102 from thecellular or telecommunications network 1000. The remote communicationsmodule 202 and the vehicle controller 106 operate together to receivethe one or more amount of data control commands.

In step 528, the vehicle controller 106 is responsive to the amount ofdata control command to adjust the amount of data being transmitted overthe cellular or telecommunications network 1000 to the remote server 900and/or the remote user device 800. For example, the controller 106 maychange any one or more of the data transmission rate, the video framerate, the size of the view of the video data, the grayscale or color ofthe video data, gradation in color or grayscale of the video data or anyother parameter relating to the amount of video data transmitted fromthe vehicle 100. The specific parameters to change may be defined by theamount of data control command. For example, a user or operator mayselect a cropped view of the video data, a reduced frame rate, etc.,which selections will be embodied in the one or more data controlcommands. Alternatively, the one or more data control commands may bemore generic and specify that data usage should be changed and thevehicle controller 106 may determine one or more data usage parametersfor change.

The present disclosure thus allows remote control of the amount of datasent over the telecommunication network 1000, thereby allowing user oroperator control of data usage as well as video quality, depending uponthe circumstance.

Smart Vehicle Video Transmission Control

In various embodiments, any one or more of the vehicle control system102, the user device control system 802 and the remote server controlsystem 902 include a video data analyzer 107 (shown in FIG. 3 to beincluded as part of the vehicle control system) configured to analyzethe video data from a plurality of cameras 103 to determine activity ina view of a subset (for example, just one) of the plurality of cameras103. For example, the video data analyzer 107 may be configured toanalyze time spaced frames of video data to determine movement in thevideo data to determine activity in the video data from a subset of thecameras 103.

In various embodiments, the vehicle controller 106 is configured to beresponsive to the video data analyzer 107 to adjust the amount of datatransmitted from the vehicle 100. For example, should no activity orsubstantially no activity be determined by the video data analyzer 107,then the controller 106 can be configured to respond by defining one ormore data transmission parameters corresponding to a relatively lowamount of video data being transmitted from the vehicle 100. Conversely,should activity be detected or determined by the video data analyzer107, then the controller 106 can be configured to respond by definingone or more data transmission parameters corresponding to a relativelyhigh amount of video data being transmitted from the vehicle 100. Anyone or more of the parameters described above for adjusting the amountof data transmitted from the vehicle 100 may be utilized. The followingexemplary parameters may be adjusted: the size of the view defined bythe transmitted video data, the frame rate of the video data, the datarate of the video data (as defined by an encoder for compressing thevideo data, for example), grayscale or color settings, and the like.

In various embodiments, the video data analyzer 107 is configured tooutput a direction or location of the determined activity. The vehiclecontroller 106 is configured to be responsive to the determination ofactivity and the direction thereof by the video data analyzer 107 and tocrop the field of view defined by the video data based thereon. In thisway, a field of view of video data may be transmitted from the vehicle100 based on a determination by the vehicle controller 106 of a relevantpart of the video data available from the one or more cameras 103.

In various embodiments, the vehicle controller 106 is configured to beresponsive to the determination of activity by the video data analyzer107 and the direction thereof by transmitting video data from one or asubset of the one or more cameras 103. The one camera 103 or the subsetof the plurality of camera 103 selected based on the field of view ofthat camera and the direction of activity determined by the video dataanalyzer 107 to ensure the relevant video data is transmitted.

For example, the vehicle 100 may comprise a plurality of cameras 103 andthe vehicle controller 106 is configured to select one or a subset ofthe a plurality of cameras 103 based on the direction of activity fromthe video data analyzer 107. Alternatively, a surround video (e.g. amosaic of video data from respective cameras 103 or stitched togethervideo data from the cameras 103) may be constructed by the vehiclecontroller 106 from the video data from the plurality of cameras 103 andthe vehicle controller 106 may determine that activity is present in oneor more parts of the video data based on the activity determination fromthe video data analyzer 107. The controller 106 is configured totransmit the video data from the selected one or the subset of cameras103 or is configured to transmit the one or more parts of the videodata, thereby sending activity relevant video data and allowing reduceddata usage.

Safety Triggered Video

In various embodiments, the one or more security sensors 166 or the oneor more event sensors 167 are configured to output a trigger signal. Theone or more security sensors 166 may comprise a door sensor configuredto output a trigger signal when one or more of the doors 101, 110 havebeen opened and the vehicle doors 101,110 are in a locked state (therebyindicating a break-in). The one or more security sensor 166 may comprisea trunk, gate or hood sensor configured to output a trigger signal whenthe trunk or gate 101, 111 has been left open. The one or more securitysensors 166 may comprise a key fob copy sensor configured to determine acounterfeit key fob. The one or more security sensors 166 may comprise aforce sensor such as a vibration sensor or such as an accelerometerconfigured to output a trigger signal when a force is applied to thevehicle 100 indicative of a potential break-in or vandalism of thevehicle 100. The one or more vehicle cameras 103 may also be comprisedin a security sensor 166. Specifically, the video data analyzer (alsoreferred to as a theft event analyzer) 107 may be configured to analyzethe video data for activity and based on a determination of durationand/or proximity of the activity be configured to output a triggersignal.

In various embodiments, the one or more event sensors 167 may alsooutput a trigger signal. The one or more event sensors 167 are operablewith an event sensing module run on the vehicle controller 106 todetermine an event (e.g. in which the vehicle contacts another vehicleor object) and to responsively output the trigger signal.

In various embodiments, the vehicle controller 106 is configured toreceive the trigger signal and respond thereto by activating the one ormore cameras 103 to capture video data (if not already activated) and toinitiate transmission of a safety event notification signal by using theremote communications module 202. Further, the one or more microphones289 may be activated to capture sound data. The vehicle controller 106is configured to transit the video data and/or the safety eventnotification signal and any sound data at least partly over thetelecommunications or cellular network 1000.

A vehicle event, attempted vandalism or theft may be determined by theone or more safety sensors 166 or the one or more event sensors 167(e.g. that may detect contact between the vehicle and another vehicle orobject). Such events are referred to herein as safety events. Thevehicle controller 106 is responsive thereto by initiating videostreaming and/or video data recording.

In various embodiments, the server controller 910 and/or the user devicecontroller 810 is/are configured to be responsive to a safety eventnotification signal by displaying a corresponding notification on thedisplay unit 814, 914. The notification on the display unit 814, 914 maycomprise an initiate video selection graphic, which when selected by wayof the input unit 816, 916, causes initiating of display of video dataon the display unit 814, 916 and play of any sound data on one or morespeakers (not shown). The video data displayed may be substantially 360°surround video (e.g. a mosaic of video data or stitched video data). Thevideo data may come from internal and/or outside video cameras 103.Further, the sound data may come from internally directed and/orexternally directed microphones.

In various embodiments, the server controller 910 and/or the user devicecontroller 810 is/are configured to initiate recording of the video datain the respective data storage device 812, 912. The sound data may alsobe recorded. The recoding of the video data may be initiatedautomatically for safety events, i.e. as a predetermined step of aprogram being run by the controller 810, 910, or it may first requireuser or operator authorization. To obtain such authorization, thecontroller 810, 910 is configured to prompt, though the display unit814, 914, a begin recording selection graphic 616 (or other selectableinput) via the input unit 816, 916.

In various embodiments, the vehicle controller 106 is configured torecord the video data (and any sound data) in the data storage device178 in response to the trigger signal. The data storage device 178 maybe a non-volatile storage device as discussed above. The data storagedevice 178 may be capable of recording or storing at least 1, 5, 10, 15or 30 minutes of the video data from the one or more cameras 103 or atleast 1 hour of the video data. The data storage device 178 is thus ableto record the video data captured over a sufficient period of timecorresponding to the vehicle safety event, e.g. vandalism, theft, otherevent, or the like.

In various embodiments, the vehicle controller 106 is configured tochange one or more parameters of the video data in response to thetrigger signal. For example, the controller 106 may be configured tochange the settings of the video data to default settings. In variousembodiments, the controller 106 is configured to set an appropriateframe rate for the video data either as captured or as transmitted. Invarious embodiments, the controller 106 is configured to set anappropriate data rate for the video data as transmitted. In variousembodiments, the controller 106 is configured to set the video data tobe captured or transmitted in color. In various embodiments, thecontroller 106 is configured to set the resolution of the video dataeither as captured or as transmitted. The various default settingsensure sufficient video data quality during a safety event so thatreal/time or subsequent display of the video data is useful inidentifying the events and the perpetrator(s).

In various embodiments, the vehicle controller 106 is configured to beresponsive to the trigger signal and to the video data analyzer 107 totransmit activity specific video data. In particular, the video dataanalyzer 107 is configured to output an indication of direction of theactivity. The vehicle controller 106 is configured to transmit videodata from one or more cameras 103, being one of a plurality of vehiclecameras 103 or a subset of the plurality of vehicle cameras 103, havinga field of view including a location of the indicated direction ofactivity. The activity specific transmission of data has been discussedin more detail above and is applicable to the present embodiments.

In various embodiments, the vehicle controller 106 is configured todetermine whether a connection to a telecommunications network 1000 isavailable in response to the trigger signal. If not, the vehiclecontroller 106 is configured to record video data captured by the one ormore cameras 103 in the vehicle data storage device 178. In embodiments,the vehicle controller 106 is configured to record the video data in thevehicle data storage device irrespective of whether a telecommunicationsnetwork 100 is available. In various embodiments, the vehicle controller106 is configured to transmit the recorded video data from the vehicle100 at least partly over the telecommunications or cellular network 1000when the connection to the telecommunications or cellular network 1000becomes available. In other embodiments, the recorded video data neednot be transmitted over the cellular network 1000 by the vehiclecontroller 106. The recorded video data can be transferred to anothercomputing device, e.g. by removing an SD card embodying the vehicle datastorage device 178, and/or viewed on the vehicle display screen 191. Thevideo data may be transmitted from the vehicle 100 to the remote server900 and/or the user device 800 either from the vehicle controller 106 orfrom the another computing device. The server controller 910 and/or theuser device controller 810 is configured to receive the video data anddisplay the video data through the display unit 814, 914. In variousembodiments, the vehicle controller 106 is configured to first transmitover the cellular or telecommunications network 1000 a safety eventnotification signal. The server controller 910 and/or the user devicecontroller 810 is configured to display a prompt for the user oroperator in response to the safety event notification signal. The promptinvites the user or operator to select to receive the video data of thesafety event. Assuming a selection is entered through the input unit816, 916 corresponding to a request to receive the video data, thecontroller 810, 910 is configured to transmit a control command to thevehicle control system 102 for transmission of the recorded video data.The vehicle controller 106 is responsive to the control command fortransmission of the recorded video data by sending the recorded videodata at least partly over the cellular or telecommunications network.

In various embodiments, recording of the video data in the vehicle datastorage device 178 may continue until a predetermined time has elapsed(e.g. 1 to 30 minutes, in one embodiment, although this may vary inother embodiments) or the vehicle controller 106 may be configured todetermine when the safety event has come to an end. For example, thevideo data analyzer 107 may be configured to determine activity asdescribed above. When a determination is made of a lack of activity, acorresponding activity output is provided to the controller 106. Thecontroller 106 is configured to discontinue recording of the video databased on the determination of a lack of activity. In other alternatives,the one or more security sensors 166 or the one or more event sensors167 may provide an output indicative of safety event end to which thevehicle controller 106 is responsive.

The video data corresponding to the safety event may be labelled as suchwhen stored to prohibit overwriting and/or to assist subsequentplayback. For example, the vehicle controller 106 may be configured torecord video data (for example, all video data) captured by the one ormore cameras 103 and to do so in a recording loop on the vehicle datastorage device 167. The vehicle controller 106 may be configured not tooverwrite the video data labelled as a safety event (or other scheme forprohibiting overwriting of a selected part of the video data). Therecording loop can, therefore, exclude the part prohibited fromrecording.

In various embodiments, the recorded video data is able to be viewed(and any sound data heard) at the vehicle 100. In particular, thecontroller 106 is configured to display the recorded video data on thevehicle display screen 191 and to output any sound data through one ormore vehicle speakers (not shown), such as on request through a userinput such as through a touchscreen input on the display screen 191. Assuch, the recorded video data can be played back at the vehicle 100, aswell as through a remote user device 800 and a remote server 900.

FIG. 7 shows a flowchart of a process according to various embodimentsof the present disclosure.

In step 540, an output trigger signal is generated. The output triggersignal may be generated by, in cooperation with the vehicle controller106, one or more security sensors 166 based on vibration,door/trunk/gate/hood open condition, activity around the vehicle 100 asdetermined by the video data analyzer 107, key copy determination, orthe like. Further, the trigger signal may be responsive to an event asindicated to the vehicle controller 106 by the one or more event sensors167. The output trigger signal may, when instigated by the one or moresecurity sensors 166, cause the vehicle's alarm system to become active,which may include a loud sound, such as a siren sound, being issuedthrough the sound device 188 and/or activation of one or moreillumination devices 284 such as flashing thereof. The output triggersignal may, when instigated by the one or more event sensors 167, causethe vehicle's event response to become active such by releasing one ormore air bags.

In steps 542 to 550, the output trigger signal also leads totransmission of video data (and optionally sound data) to a user device800 and/or a remote server 900 so that the safety event can be viewed inreal time or substantially real time when a connection is available andso that a recording of the safety event can be played back when aconnection becomes available.

In step 542, a determination is made, via the vehicle controller 106, asto whether a connection to a telecommunications or cellular network 1000is available. If a connection is available, then the process proceeds tostream the video (and optionally) data according to steps 544 to 546. Ifno connection is available, then the process proceeds to record thevideo (and optionally sound) data and subsequently upload the video datawhen the connection to the telecommunications or cellular network 1000becomes available. The recording of video data may proceed withoutdependence on determination of no connection being available.

In step 544, when a connection to the network 1000 is available, thevehicle controller 106 is configured to transmit, using the remotecommunications module 202, a safety event notification to the remoteserver 900 and/or the user device 800. The remote server controller 910and/or the user device controller 810 is/are configured to receive thesafety event notification and to display an invitation to the user oroperator to accept a video streaming request relating to the safetyevent. Video data may be recorded in the data storage device 178, 812,912 of any one of the vehicle 100, the remote server 900 and the userdevice 800 in addition to the video data being streamed.

When the user or operator accepts the video streaming request, andaccording to step 546, the vehicle controller 106 is configured tostream video data captured by the one or more vehicle cameras 103 to theremote server 900 and/or the user device 800.

The server controller 910 and/or the user device controller 810 areconfigured to receive the video data and display the streamed video dataon the display unit 814, 914. Sound data may also be replayed. The useror operator is thus able to view the safety event in real time. Further,the user or operator can trigger control commands through the input unit816, 916 as discussed above and as detailed further with reference toFIG. 6. For example, control commands corresponding to sound activation,low light enhancement, illumination activation, video data recordactivation, camera wash activation, camera articulation activation mayall be useful. Further, video data corresponding to the safety event maybe stored in the data storage device 178, 812, 912 of any one of thevehicle 100, the remote server 900 and the user device 800. The videodata corresponding to the safety event may be labelled as such whenstored to prohibit overwriting and/or to assist subsequent playback.

If a determination is made that no connection is available in step 542,the vehicle controller 106 is configured to commence recording videodata (and sound data) in step 550. The video data continues to berecorded until safety event end time. The safety event end time may bedetermined by a predetermined time having elapsed from the triggersignal or until the video data analyzer 107, or the sensors 166, 167indicate that activity has finished. A safety event occurs from thetrigger signal to the safety event end time. The recorded video data maybe labelled in the data storage device 178 as a safety event so that thevehicle controller 106 is able to prohibit overwriting of the safetyevent video data. The step 550 of recording video data may take placeindependent of the determination of whether a connection is available.

Optionally, the process continues to check for a network connection instep 552. Once a connection is available, the video data including thesafety event, and labelled as such in the data storage device 178, isuploaded or otherwise transmitted to the remote server 900 or the userdevice 800 in step 556. The video data is transmitted at least partlyover the cellular or telecommunications network 1000 from the vehicle100. The recorded video data need not necessarily be subsequenttransmitted to the remote server 900 by the vehicle controller 106.

Before transmitting or uploading the video data in step 556, a safetyevent notification signal may first be transmitted over the cellular ortelecommunications network 1000 in step 554. The safety eventnotification 554, when received by the server controller 910 or the userdevice controller, may cause the respective controller 810, 910 todisplay a prompt on the display unit 814, 914. The prompt may comprisean invitation to the user or operator to request, through the input unit816, 916, to upload or other transmission of the video data.

In step 558, the video data transmitted to the remote server 900 and/orthe user device 800 is displayed on the display unit 814, 914 under thecontrol of the controller 810, 910.

The present disclosure thus allows prompt notification, a record of andreal-time view of theft and vandalism attempts and/or other vehicleevents that are activated in an automated, sensor initiated way. Thesefeatures may assist in reducing crime and identifying causes of vehicleevents.

User (Operator) Interface

The user device control system 802 and/or the server control system 902may include a program, e.g. an app, stored on the respective memory 908,808. The program, when run by the controller 910, 810 is able toimplement the various functions of the remote server 900 and the remoteuser device 800 that have been described above, particularly withreference to FIGS. 4, 5 and 7, such as receiving video data andnotifications from the vehicle 100, recording the video data, generatingcontrol commands and transmitting the control commands to the vehicle100. The app is also able to implement the further server and/or userdevice aspects of the present disclosure detailed below with referenceto FIG. 10. Further, the program is configured to generate a graphicaluser interface 600 that is displayed on the display unit 814, 914 and tohandle operator and/or user inputs through the input unit 816, 916,thereby allowing the control command selections to be made that havebeen described in the foregoing. It is to be understood that the inputof control commands by using a graphical user interface 600, asdescribed below, is merely exemplary. One, some or all of the inputsallowed by the graphical user interface 600 described below may beachieved by other input devices such as mechanical buttons.

An exemplary graphical user interface 600 is shown in FIG. 6, which maybe displayed on the display unit 814, 914 of the remoter server 900 orthe user device 800. This graphical user interface 600 is particularlydescribed with respect to the remote server 900 or the user device 800.However, such a graphical user interface 600, and the various controlfunctions allowed by it, may also be included within the vehicle 100,and displayed on the vehicle display screen 191.

In various embodiments, the graphical user interface 600 includes avideo display portion 621. The video display portion 621 constitutes amajor part of the graphical user interface 600 and defines an area inwhich the video data captured by the one or more vehicle cameras 103 isdisplayed.

In various embodiments, the graphical user interface 600, generated bythe program run on the controller 810, 910, is able to displaysubstantially 360° surround video data as a mosaic of videos fromrespective cameras 103 or a video stitched together from the video dataof each camera 103, as has been indicated in the foregoing. A mosaic ofvideos can be displayed in the display portion 621 by way of a splitscreen, with each part of the display portion displaying video data froma respective camera 103. The display portion 621 may show video datafrom respective cameras 621, with surround video data able to be viewedby successively showing video data from different camera 103. Togglingbetween camera views may be carried out automatically by the controller810, 910 or it may be user or operator controlled as described below.Alternatively, the display portion 621 may show a stitched togetherdisplay of video data from plural cameras 103.

The graphical user interface 600 comprises one or more selectiongraphics 602, 604, 606, 608, 610, 612, 614, 616 that can be selectedthrough the input unit 816, 916. The selection graphics 602, 604, 606,608, 610, 612, 614, 616 allow for user or operator selection of controlcommands to be generated. The input unit 816, 916 may be embodied as amouse, a keyboard, a touchscreen, and the like.

In various embodiments, one or more directional selection graphics 602are provided as part of the graphical user interface 600. In the shownembodiment, the directional selection graphics 602 are provided as oneor more arrows 602. However, other graphics are possible such as one ormore scroll bars. The one or more directional selection graphics 602allow, in one use, a user or operator to control a viewing directionwithin a substantially 360° surround video sent from the vehicle 100 anddisplayed in the display portion 621. The controller 810, 910 isresponsive to a selection of the one or more directional selectiongraphics 602 to change the view accordingly in the display portion 621of the graphical user interface 600. The directional selection graphics602 may allow changing view within a stitched together panoramic videofrom plural cameras 103 or toggling between respective camera views.

In various embodiments, a camera wash or wipe selection graphic 604 isincluded in the graphical user interface 600. The camera wash selectiongraphic 604 allows a user or operator to select generation of a camerawash or wipe command as discussed above.

In various embodiments, the graphical user interface 600 includes acamera change selection graphic 610 corresponding to the controller 810,910 displaying in the display portion 621 a view from one or a subset ofa plurality of the vehicle cameras 103. The camera change selectiongraphic 610 may include a plurality of further selection graphics (notshown) each corresponding to a particular one or subset of the pluralityof vehicle cameras 103. In one possibility, the vehicle control system102 is configured to transmit video data from each of the plurality ofcameras 103. In this possibility, the server or user device controller810, 910 is configured to display the camera view according to the useror operator selection of the camera change selection graphic 610 throughthe input unit 816, 916. In another possibility, the server or userdevice controller 810, 910 is configured to transmit a camera changecommand depending on the user or operator preference selected using thecamera change selection graphic 610 and the input unit 816, 916. Thevehicle control system 102 is configured to respond to the camera changecommand and transmit video data according to the selected camera 103, asdescribed above.

In various embodiments, an illumination device selection graphic 608 isincluded in the graphical user interface 600. The illumination deviceselection graphic 608 allows a user or operator to select to have thecontroller 810, 910 generate an illumination device control command asdiscussed above. The illumination device selection graphic 608 isconfigured to allow an illumination device ON command and optionallyalso an illumination device OFF command. The illumination deviceselection graphic 608 may also include a plurality of further graphics(not shown) corresponding to each of a plurality of vehicle illuminationdevices 284 to allow for one or a subset of each of the illuminationdevices 284 to be turned ON and optionally also OFF depending on theuser or operator selection through the input unit 816, 916. Thecontroller 810, 910 is configured to generate an illumination device 284specific control command corresponding to the selection made. The ON andOFF control commands have been described further above.

In various embodiments, a low light enhancement selection graphic (notshown) is included. The low light enhancement selection graphic allows auser or operator to select to have the controller 810, 910 generate alow light enhancement control command as discussed above.

In various embodiments, a camera articulation selection graphic 606 isincluded in the graphical user interface 600. The camera articulationselection graphic 606 corresponds to the controller 810, 910 generatinga camera articulation control command as discussed above, when selectedthrough the input unit 816, 916. The camera articulation selectiongraphic 606 may include a plurality of further selection graphics (notshown) that allow a user to select a camera 103 for articulation and/ora direction of articulation (e.g. left or right or up or down). Thecontroller 810, 910 is configured to generate a control commandcorresponding to the articulation selection made.

In various embodiments, a sound activation selection graphic 612 isincluded as part of the graphical user interface 600. The soundactivation selection graphic 612, when selected through the input unit816, 916, corresponds to the controller 810, 910 generating a soundsactivation control command as described above.

In various embodiments, the graphical user interface 600 includes awake-up or video ON selection graphic 614. When the wake-up or video ONselection graphic 614 is selected through the input unit 816, 916, theremote server controller 910 or the user device controller 810 isconfigured to generate and transmit a wake-up or video ON controlcommand to the vehicle control system 102, thereby initiatingtransmission of video data from the vehicle 100 to the remote server 900or user device 800. The wake-up or video ON selection graphic 614 mayalso be deselected to allow a user or operator to turn OFF transmissionof video data from the vehicle 100.

In various embodiments, the graphical user interface 600 includes avideo size selection graphic 618. The video size selection graphic 618may include a plurality of further graphics to allow the size of thefield of view to be selected. The video size selection graphic 618 isoperable to allow a user or operator to select the size of the field ofview through the input unit 816, 916. The server controller 910 or theuser device controller 810 is configured to receive the user or operatorselection and generate a corresponding control command as describedabove.

In various embodiments, the graphical user interface 600 includes arecord selection graphic 616, which, when selected through the userinput 816, 916, begins or ends recoding of video data in the datastorage device 812, 912.

It will be appreciated that the user interface 600 may include one, someor all of the various graphics 602 to 618. Further, although describehere as a graphical user interface, other user interfaces are possiblesuch as command line, drop down box, etc. The user may be presented withvarious control selection possibilities through other non-graphical userinterfaces.

In various embodiments, the user interface of the present disclosureallows a number of control commands to be transmitted through a remoteuser device 800 or a remote server 900 to implement many usefulfunctions described herein, including data usage control, securityfeatures control, video quality control, view selection, etc. Thesefunctions may be achieved through a coherent and intuitive userinterface.

Video Transmission Button

In various embodiments, a video transmission button 624 is provided thatallows for a video transmission, which may be included as part of aconference, to an operator of the remote server 900. The videotransmission button 624 may also allow for transmission of sound datacaptured by the one or more microphones 289. The button 624 may beincluded as part of the vehicle 100 or as part of a key fob or key fobapp. The video transmission button 624 utilizes video data from one ormore cameras 103 as the source of video data for the video transmission.The one or more cameras 103 may comprise one or more cameras 103 havingthe outside surroundings in their field of view and may optionallyinclude a camera 103 having the interior of the vehicle 100 in its fieldof view. The one or more microphones 289 may also capture sound datafrom the outside surround and/or the interior of the vehicle 100. Thevideo data for the video transmission may constitute substantially 360°surround video data or otherwise video data patched together from morethan one camera 103. The video transmission button 624 may be useful inthe event of an event, theft or other emergency situation to allow anoperator at the remote server 900 to assist one or more occupants of thevehicle with provision of emergency and other services.

In one example shown in FIG. 9, the rear view mirror 140 includes thevideo transmission button 624, which is an emergency button in thisembodiment. The rear view mirror can, but need not, also include any oneof a camera 103 for capturing video of the interior of the vehicle 100(and particularly of the driver), a non-emergency conference button 622and a call button 620. The emergency video transmission button 624places a priority connection that is given higher priority at the remoteserver 900 than a connection initiated by the non-emergency conferencebutton 622. The non-emergency conference button 622 allows for a lowerpriority conference with an operator of the remote server 900, who mayprovide turn by turn directions and other assistance. The call button620 allows various voice commands to be communicated to the vehiclecontrol system 102 to contacts other than operators of the remote serer900.

The buttons 620, 622, 624 may be in wired communication with thecontroller 106. The buttons, when selected by a user, may generate acorresponding signal that is sent to the controller 106. The controller106 is configured to respond to the signal by executing the associatedfunction such as placing a conference call with an operator of theremote server 900 or placing some other telephone call.

The arrangement of buttons 620, 622, 624 on the rear view mirror 140 ismerely exemplary. The one or more buttons 620, 622, 624, particularlythe emergency button 624, may be located elsewhere in the vehicle 100such as on a dashboard or as a graphic on the display screen 191.

Additionally or alternatively, the emergency button 624 may be includedon as a button on a key fob or on a key fob app 640, as shown in FIG.10. In FIG. 10, the key fob or key fob app includes the emergency button624, thereby allowing video transmission to an operator (which may beincluded as part of a conference call with the operator) of the remoteserver 900 to be initiated from outside of the vehicle 100. The key fobor key fob app 640 may include any one or more of the following furtherbuttons: ignition start and optionally stop buttons 626, 628, door lockand unlock buttons 630, 632, sound and light activation and deactivationbuttons 634, 636.

In the example of a key fob, the buttons 624 to 636 may be provided asmechanical buttons. The key fob 640 may be configured to generate awireless signal representative of a button pressed. The wireless signal,such as an infrared signal, is receivable by the vehicle controller 106via the remote communications module 202. The vehicle controller 106 isconfigured to execute a control operation to perform the operationassociated with the button pressed such as unlocking the doors 101, 110,starting the ignition, switching on the sound device 188 or the one ormore illumination devices 284, 286.

In the example of a key fob app, the buttons 624 to 636 may be providedas a graphical button displayed on a remote and portable user device800, such as a smart mobile telephone or tablet. The controller 810 ofthe user device 800 is configured to generate and transmit a wirelesssignal representative of one of the buttons 624 to 636 selected by auser. The buttons 624 to 626 may be selected through the input unit 816in the form of a touchscreen. The user device controller 810 isconfigured to generate a wireless signal, either in the form of directline of communication such as an infrared signal, or through thetelecommunications or cellular network 1000, which is receivable by thevehicle controller 106 via the remote communications module 202. Thevehicle controller 106 is configured to perform a control operation toexecute the function associated with the received wireless signal suchas unlocking the doors 101, 110, starting the ignition, switching on thesound device 188 or the one or more illumination devices 284, 286.

In various embodiments, when the video transmission button 624 (whichmay be an emergency button) is selected by a user or operator, acorresponding signal is sent to the vehicle controller 106. The vehiclecontroller 106 is configured to capture video from one or more cameras103 (one or more outside cameras 103 and/or one or more interior cameras103) and stream or otherwise transmit the captured video (and any sounddata captured by the one or more microphones 289) to the remote server900 at least partly over the telecommunications or cellular network 1000via the remote communications module 202. The vehicle controller 106 maybe further configured to open a two-way audio channel at least partlyover the cellular or remote telecommunications network 1000 to allow theoperator and one or more occupants of the vehicle 100 to talk to oneanother. The transmission of video data and the two-way audio channelprovides for a conference between the operator and the vehicle occupantthat includes video data. Sound data from outside the vehicle 100 mayalso be transmitted. It is envisaged that two-way video transmissioncould also be established such that the operator may be viewed in thevehicle display 191.

The remote server 900 is configured to display the video data andoptionally to play the audio through one or more speakers (not shown)such that the operator can take part in the conference. The operator maybe able to see the vehicle surroundings and/or the vehicle interiorthrough the video transmission displayed on the display unit 814, whichmay be enough to be able to establish the assistance required. Thus,even if the emergency button 624 is selected from outside the vehicle100, e.g. through the key fob or key fob app 640, the video transmissionmay be sufficient to establish that emergency services are required. Infact, an audio and video conference may not be required. Meretransmission of the video data captured by the one or more cameras 103and communicated to the remote server 900 may be sufficient to alert theoperator to the service needed.

In various embodiments in which the emergency button 624 is provided aspart of a key fob app run on the user device 800, the video data may betransmitted from the vehicle 100 as explained above, but the conferencebetween the operator and the user may take place through the userdevice. That is, the two-way audio channel is connected between the userdevice 800 and the remote server, optionally over the cellular ortelecommunications network 1000 and the video transmission is connectedbetween the vehicle 100 and the remote server 900.

In various embodiments, the vehicle controller 106 is configured tostore the captured video data (and any sound data) in the data storagedevice 178 in response to the video transmission button 624 beingselected.

In various embodiments, the remote server control system 902 isconfigured to store on the data storage device 912 the video datatransmitted in response to the video transmission button 624 beingselected. Any audio transmissions may also be stored on the data storagedevice 912 including sound data from the conference between operator anduser and/or sound data from the vehicle's outside surroundings.

The remote server 900 is also configured to generate and transmitoperator selected control commands, as described above.

FIG. 11 shows a flowchart describing a process for transmitting videodata to a remote server 900 in response to a video selection button 624being selected according to various embodiments.

In step 560, a user selects the video transmission button 624. The videotransmission button 624 may be included on a key fob 640, a key fob app640 or in the vehicle 100, as described above.

In step 562, a video transmission request signal is generated andtransmitted to the vehicle controller 106 in response to the videotransmission button 624 being selected. The signal may be transmittedvia wired connection, line of sight wireless connection (e.g. infrared)or telecommunications network connection.

In step 564, the video transmission request signal is received by thevehicle controller 106, via the remote communications module 202. Thevehicle controller 106 responds to the received signal by operating theone or more cameras 103 to capture video data and by streaming the videodata over the telecommunications network 1000 via the remotecommunications module 202.

In step 566, the video data is received by the remote server 900 via theremote communications module 904. The remote server 900 displays thereceived video data on the display unit 914, which is viewed by theoperator. The operator may be able to control the video data using anyone of the control selections shown in FIG. 6 and discussed above. Forexample, when plural camera views are streamed, such as 360° video data,the operator is able to change the camera view.

In step 568, the vehicle occupant may also be placed in conference withthe operator via establishment of a two way audio channel. To do so, thevehicle 100 includes one or more interior speakers (not shown) and amicrophone (not shown) and the remote server includes one or morespeakers (not shown) and a microphone (not shown) for the operator. Theoperator is thus able to have an interactive conversation with thevehicle occupant, as well as to view the vehicle's outside surroundingsand/or interior through the streamed video data. The operator is able totake suitable action based on the video data and the optional conferencecall. For example, the appropriate emergency services can be informed.The GPS of the vehicle 100 is also communicated to the remote serversuch that this information can be passed on to the emergency service(s).

In step 570, the remote server control system 902 and/or the vehiclecontrol system 102 is configured to record the streamed video data inthe respective data storage device 178, 912.

The various aspects of the present disclosure are combinable in anycombination. Thus, any one or more of the following aspects of thepresent disclosure may be combined: the remote video access and control,the smart vehicle video transmission control, the alarm triggered video,the user interface, and the video transmission button.

In various embodiments, when no network connection is available, thevehicle controller 106 is configured to record the video data in thevehicle data storage device 178 and later transmitted to the remoteserver 900 when a telecommunications or cellular network connectionbecomes available. This aspect of the present disclosure has beendiscussed above with respect to FIG. 7. The operator of the remoteserver 900 is able to playback the recorded video data as has beenpreviously described.

It will be appreciated that the disclosed methods, systems, and vehiclesmay vary from those depicted in the Figures and described herein. Forexample, the vehicle 100, the control system 102, and/or variouscomponents thereof may vary from that depicted in FIG. 1 and describedin connection therewith. It will similarly be appreciated that thegraphical user interface may differ from that depicted in FIG. 6. Also,the arrangement of in-vehicle buttons 620 to 624 may differ from thatdepicted in FIG. 9. Further, it will be appreciated that the key fob orkey fob app may differ from that shown in FIG. 10. In addition, it willbe appreciated that certain steps of the processes may vary from thosedepicted in FIGS. 4, 5, 8 and 11 and/or described above in connectiontherewith. It will similarly be appreciated that certain steps of themethods described above may occur simultaneously or in a different orderthan that depicted in FIGS. 4, 5, 8 and 11 and/or described above inconnection therewith.

While at least one exemplary embodiment has been presented in theforegoing detailed description, it should be appreciated that a vastnumber of variations exist. It should also be appreciated that theexemplary embodiment or exemplary embodiments are only examples, and arenot intended to limit the scope, applicability, or configuration of thedisclosure in any way. Rather, the foregoing detailed description willprovide those skilled in the art with a convenient road map forimplementing the exemplary embodiment or exemplary embodiments. Itshould be understood that various changes can be made in the functionand arrangement of elements without departing from the scope of theappended claims and the legal equivalents thereof.

The invention claimed is:
 1. A method of transmitting video dataassociated with a vehicle, the method comprising: determining, via avehicle processor, a vehicle security event based on one or moresecurity sensors, the vehicle security event pertaining to theft orvandalism pertaining to the vehicle; activating, via the vehicleprocessor, one or more vehicle cameras that capture video data inresponse to the determined vehicle security event; recording, via thevehicle processor, the video data in a vehicle data storage device uponoccurrence of the vehicle safety event; and if the vehicle processordetermines that a suitable connection to a telecommunications network isavailable when the video data is being captured, then, upon theoccurrence of the vehicle safety event, transmitting, via the vehicleprocessor, the video data over the telecommunications network insubstantially real time to a remote user of the vehicle that is remotefrom the vehicle.
 2. The method of claim 1, wherein the one or moresecurity sensors comprise at least one of a force sensor, a door sensor,a trunk sensor, a hood sensor and a key copy sensor.
 3. The method ofclaim 1, further comprising at least one of: activating a vehicle alarmincluding noise and/or light in response to the vehicle security event.4. The method of claim 1, wherein the transmitting comprisestransmitting the video data to a remote computing device, wherein theremote computing device displays, via a processor and a display unit,the video data for the remote user.
 5. The method of claim 4, whereinthe method comprises transmitting a notification to the remote computingdevice concerning the vehicle security event and requesting acceptanceof transmission of the video data, wherein the video data transmissionstep is performed after acceptance has been returned from the remotecomputing device to the vehicle processor.
 6. The method of claim 1,wherein the recorded video data stored in the vehicle data storagedevice is labelled to indicate video data relating to the vehiclesecurity event, and the method further comprises prohibiting overwritingof the video data so labelled.
 7. A vehicle control system fortransmitting video data associated with a vehicle, the control systemcomprising: one or more security sensors responsive to a vehiclesecurity event, the vehicle security event pertaining to theft orvandalism pertaining to the vehicle; and a vehicle processor configuredto: activate one or more vehicle cameras to capture video data inresponse to the vehicle security event; record the video data in avehicle data storage data device, upon the occurrence of the vehiclesafety event; and if the processor determines that a suitabletelecommunications network connection is available when the video datais being captured, then the vehicle processor is configured to transmit,upon the occurrence of the vehicle safety event, the video data over thetelecommunications network in substantially real time to a remote userof the vehicle that is remote from the vehicle.
 8. The vehicle controlsystem of claim 7, wherein the one or more security sensors comprise atleast one of a force sensor, a door sensor, a gate sensor, a hoodsensor, and a key copy sensor.
 9. The vehicle control system of claim 7,wherein the vehicle processor is configured to activate a vehicle alarmincluding noise and/or light in response to the vehicle security event.10. The vehicle control system of claim 7, wherein the vehicle processoris configured to transmit a notification to a remote computing deviceconcerning the vehicle security event and requesting acceptance oftransmission of the video data, and wherein the vehicle processor isconfigured to await receipt of the request acceptance from the remotecomputing device and thereafter to transmit the video data.
 11. Avehicle, comprising: one or more vehicle cameras; one or more securitysensors responsive to a vehicle security event, the vehicle securityevent pertaining to theft or vandalism pertaining to the vehicle; avehicle data storage data device; a vehicle control system fortransmitting video data associated with the vehicle, the control systemcomprising a vehicle processor configured to: activate the one or morevehicle cameras to capture video data in response to the determinedvehicle security event; record the video data in the vehicle datastorage data device, upon the occurrence of the vehicle safety event;and if the processor determines that a suitable telecommunicationsnetwork connection is available when the video data is being captured,then the vehicle processor is configured to transmit, upon theoccurrence of the vehicle safety event, the video data over thetelecommunications network in substantially real time to a remote userof the vehicle that is remote from the vehicle.
 12. The vehicle of claim11, wherein the one or more security sensors comprise at least one of aforce sensor, a door sensor, a hood sensor, a gate sensor, and a keycopy sensor.
 13. The vehicle control system of claim 11, comprising atleast one of a vehicle alarm including a noise device, a light device,or both and an air bag, wherein the vehicle processor is configured toactivate the vehicle alarm in response to the vehicle security event,and wherein the vehicle processor is configured to release the air bagin response to the vehicle security event.
 14. The vehicle of claim 11,wherein the vehicle processor is configured to transmit a notificationto a remote computing device concerning the vehicle security event andrequesting acceptance of transmission of the video data, and wherein thevehicle processor is configured to await receipt of the requestacceptance from the remote computing device and thereafter to transmitthe video data.
 15. The method of claim 1, wherein the vehicle securityevent pertains to theft or vandalism pertaining to the vehicle while thevehicle is parked.
 16. The method of claim 1, further comprising:modifying, via the vehicle processor, a frame rate for the video data tobe transmitted to the remote user, based on the occurrence of thevehicle safety event.
 17. The method of claim 1, further comprising:modifying, via the vehicle processor, a data rate for the video data tobe transmitted to the remote user, based on the occurrence of thevehicle safety event.
 18. The method of claim 1, further comprising:modifying, via the vehicle processor, a color for the video data to betransmitted to the remote user, based on the occurrence of the vehiclesafety event.
 19. The method of claim 1, further comprising: modifying,via the vehicle processor, a resolution for the video data to betransmitted to the remote user, based on the occurrence of the vehiclesafety event.
 20. The method of claim 1, further comprising: modifying,via the vehicle processor, each of the following: a frame rate, a datarate, a color, and a resolution for the video data to be transmitted tothe remote user, based on the occurrence of the vehicle safety event.